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EVALUATION  OF  AFTER  ACTION  REVIEW 
TAKENOME  PACKAGES 

A  UDRI  “Seardi  and  Summary”  Technical  Rqxnt 

1.  INTRODUCTION 


1.1  Background 

According  to  current  training  doctrine,  the  Army  must  “train  as  it  fights.”  To  accomplish  this  goal,  the 
Army  is  training  its  force  in  the  most  realistic  and  cost-effective  manner  possible.  An  integral  part  of  the 
training  is  the  After  Action  Review  (AAR)  which  provides  training  units  an  opportunity  to  review 
successes  and  areas  needing  improvement  after  a  training  scenario.  AARs  allow  trained 
observer/controllers  (0/Cs)  and  the  AAR  leader  to  conduct  a  Socratic  learning  session  following  training 
exercises  to  facilitate  participants  learning  from  their  exercise  experience — ^to  understand  which  tasks 
were  accomplished  according  to  standards  and  those  that  were  not  and  why.  The  AAR  is  an  integral  part 
of  the  training  to  ensure  that  the  lessons  learned  during  the  exercise  are  effectively  presented  and 
corrective  actions  are  taken.  The  overall  philosophy  is:  “Body  bags  is  not  the  way  to  learn— let  us  learn 
now  from  each  other.”  (Combat  Maneuver  Training  Center  [CMTC],  1998)  The  effectiveness  of  certain 
AAR  processes  has  been  widely  recognized  by  the  Army. 

After  Action  Reviews  (AARs)  promote  Army  readiness.  In  recent  years, 
operational  AARs  have  been  refined  and  matured  at  the  Combat  Training 
Centers  (CTCs)  and  Battle  Command  Training  Program  (BCTP).  AARs 
properly  performed  in  a  non-attribution  environment  where  pride  is  left 
at  the  door  minimize  doubt  in  the  leader's  mind  as  to  why  operations 
succeeded  or  failed.  Tactics,  force  structure,  technology,  and  weapons  in 
addition  to  warfighting  skills  have  all  benefited  from  the  AAR. 

Doctrinally,  the  AAR  is  now  an  integral  part  of  military  training. 

(TRADOC  Project  Integration  Office-Synthetic  Environment,  1997,  p  1) 

1 .2  Take-Home  Package  (THP)  Definition 

Like  the  AAR,  the  THP  is  an  essential  part  of  the  overall  exercise.  The  THP  reinforces  lessons  learned 
from  the  exercise  and  discussed  in  the  AAR.  THPs  also  provide  commanders  with  a  plan  to  develop  home 
station  training.  The  THP  is  a  tool  that  allows  units  to  continue  learning  from  a  previous  exercise.  In 
essence  the  THP  is  a  compilation  of  the  body  knowledge  presented  in  the  AAR  with  the  addition  of 
specific  comments  from  the  0/Cs,  present  during  the  exercise.  “The  imit  is  provided  with  take-home 
materials,  which  include  the  0/C  comments,  umt  performance  data,  and  videotapes  of  the  after  action 
reviews.  This  take-home  package  will  be  used  at  the  home  station  to  plan  future  (1  year  -  18  months) 
training.”  (Joint  Readiness  Training  Center  (JRTC),  1998a) 
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1.3  THP  Contents 


THPs  consist  of  information  and  an  expected  review  process  used  by  the  military  in  an  attempt  to  extend 
the  usefulness  of  the  lessons  learned  from  unit  training  exercises.  Currently  no  standardized  system  exists 
for  developing  AARs  and  THPs,  as  indicated  in  the  STAARS  Handbook  version  2.1  (Department  of  the 
Army  (DA),  1997a).  This  lack  of  standardization  has  resulted  in  several  different  data  formats  for  THPs 
being  used.  At  one  end  of  the  spectrum  THPs  can  be  as  simple  as  a  Microsoft  Excel  spreadsheet 
containing  information  generated  from  0/C  notes,  taken  while  a  unit  completes  an  assigned  series  of 
training  tasks,  also  known  as  a  training  table.  At  the  other  end  of  the  spectrum,  the  newer  training  systems 
(live  and  simulation)  are  capable  of  automatically  compiling  most  of  Ae  relevant  data  collected  during 
training  exercises  into  AAR  and  THP  products.  These  data  may  include  video  clips,  radio  transmissions, 
times,  and  relative  locations  of  component  units.  This  type  of  computer-generated  and  tracked  data  is 
generally  found  in  THPs  from  simulation  exercises.  The  Array  is  attempting  to  make  this  type  of  data 
available  to  the  participants  of  live-exercises  through  several  different  programs  such  as  the  following: 

•  SINGARS  Monitor  and  Control  Subsystem  “This  subsystem  will  monitor,  collect,  time 
tag,  and  store  radio  traffic  for  incorporation  into  AARs  and  THPs.”  (STRICOM,  1998) 

•  The  SAWE-RF/MILES 11  (Simulated  Area  Weapons  Effects  -  Radio  Frequency/Multiple 
Integrated  Laser  Engagement  11)  incorporates  Global  Positioning  System  (GPS)  data  with  the 
Core  Instrumentation  System  (CIS)  data  to  provide  real-time  battle  tracking  and  casualty 
assessment,  as  well  as  having  the  advantage  of  live  video  feeds  from  the  battlefield  for  use  in 
AARs  and  THPs.  (CMTC,  1998) 

•  The  Joint  Readiness  Training  Center  (JRTC),  Ft.  Polk.  The  JRTC  Instrumentation  System 
will  provide  for  real-time  video,  position  location,  and  small  arms  engagement  data  to 
analysts  and  the  observer  controllers.  The  result  will  be  accurate  and  immediate  data  to 
provide  to  the  analysts  for  improved  AARs  and  unit  THPs.”  (JRTC,  1998b) 

1.4  Purpose  and  Scope 

THPs  capture  the  tactical  essence  of  our  Army,  providing  snapshots  of 
the  state  of  training.  In  providing  unit  feedback,  no  other  method  has 
been  as  successful  in  giving  the  Army  a  clear  assessment  of  its  potential 
(Scudder,  1994,  p  31). 

Although  Army  leaders  recognize  the  potential  value  of  THPs,  there  seems  to  be  considerable  discussion 
as  to  their  usefulness  (Crissey,  Dr.  Mona,  personal  communication,  August  6, 1998).  Therefore, 
Simulation,  Training  and  Instrumentation  Command  (STRICOM)  requested  UDRI  to  conduct  a  Search 
and  Summary  to  determine  whether  the  effectiveness  of  THPs  has  been  evaluated  in  the  scientific  and 
technical  literature. 

1.5  Overview 

After  reviewing  the  approach  used  in  this  Search  and  Summary,  this  report  highlights  typical  THPs,  their 
contents,  and  differences  in  content  with  respect  to  organizational  level,  and  live  versus  simulation 
exercises.  Then  it  presents  representative  AAR  and  THP  systems  that  illustrate  current  capabilities. 
Finally,  the  report  concludes  with  a  summary  of  findings  for  all  identified  literature  that  presents 
evaluations  of  THPs. 
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2.  APPROACH 

The  University  of  Dayton  Research  Institute  (UDRI)  Human  Factors  Group  conducted  an  extensive 
search  of  military,  government,  commercial,  and  World  Wide  Web  (WWW)  databases  to  locate  resources 
related  to  the  effectiveness  of  military  training  THPs.  The  formal  search  consulted  numerous  government 
and  commercial  databases.  Additionally,  searches  were  conducted  via  the  World  Wide  Web  to  locate 
resources  not  available  in  the  aforementioned  databases.  The  scope  of  this  preliminary  study,  however, 
did  not  allow  for  resources  to  conduct  interviews  or  surveys  of  the  creators  and  end-users  of  THPs. 

2.1  Literature  Search  Process 

The  main  thrust  of  this  project  consisted  of  searching  available  resources  for  items  related  to  THPs  and 
their  effectiveness.  Toward  this  end,  we  conducted  a  search  via  the  Defense  Technical  Information  Center 
(DTIC)  and  the  web  to  find  any  literature  available.  Since  AAR  technology  has  changed  so  much  within 
the  last  eight  years,  our  literature  search  focused  on  documents  dated  1987  or  later,  though  an  occasional 
reference  is  made  to  earlier  documents.  A  complete  listing  of  the  databases  search  for  this  project  are 
listed  below;  however  all  pertinent  citations  came  fi'om  the  DTIC  family  of  databases  and  WWW 
searches. 

■  DTIC  CD-ROM 

■  DTIC  Defense  RDT&E  Online  System  (DROLS) 

■  National  Aeronautics  and  Space  Administration  (NASA)  Recon 

■  iDialog,  and 

■  PsycINFO 

■  Education  Abstracts 

■  Dissertation  Abstracts 

■  Compendex 

In  addition  to  tiiese  databases  UDRI-HFG  searched  the  WWW  by  querying  most  of  the  established 
Internet  search  engines,  using  the  keywords  listed  below. 

2.2  Keywords 

The  initial  search,  conducted  in  November  1998  by  UDRI  analysts  and  the  UDRI  Technical  Information 
Service  Office,  focused  on  identifying  documents  concerning  military  training  AAR  THPs  and  then- 
effectiveness.  Keywords  used  include  the  following; 

•  Debriefing  or  Critique 

•  After  Action  Review  or  AAR 

•  Take-home  Report,  Take  Home  Report,  Take  Home  Package  or  Take-Home  Package  (THP) 
(Note:  used  both  the  unit  modifier  “take-home”  and  separate  words  “take  home”) 

•  Performance  assessment 

•  Evaluation  report 

•  Performance  critique 

•  Effectiveness  (of  THPs) 

Analyzing  the  results  from  the  initial  search  allowed  us  to  narrow  the  search.  This  resulted  in  searches  on 
specific  US  Army  training  systems  and  any  studies  conducted  in  the  AAR  area.  Once  ftie  information 
was  taken  from  the  initial  search,  we  concentrated  on  specific  systems  within  the  US  Military,  focusing 
primarily  on  the  Army,  such  as  the  following  (presented  alphabetically): 
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•  Automated  Training  Analysis  and  Feedback  System  (ATAFS)* 

•  Aviation  Reconfigurable  Manned  Simulator  System  (ARMS) 

•  Battle  Command  Training  Program  (BCTP) 

•  Brigade  Operation  Display  and  After  Action  Review  System  (BODAS) 

•  Combat  Training  Center  Archive 

•  Combatant  Command  Training  (CTC) 

•  Corps  Battle  Simulation  After  Action  Review  System  (CBS  AARS) 

•  Digital  After  Action  Review  Technology  (DAART) 

•  Joint  Task  Force  Training  Program  (JTFTP) 

•  National  Training  Center  -  Objective  Instrumentation  System  (NTC-OIS)* 

•  Simulation  Training  Integrated  Performance  Evaluation  System  (STRIPES) 

•  Standard  Training  Assessment  &  Reporting  System  (STARS) 

•  Standardized  Army  After  Action  Review  System  (ST AARS) 

•  Training  Analysis  Repository  and  Graphical  Evaluation  Toolset  (TARGET)* 

•  Unit  Performance  Assessment  System  (UPAS)* 

Since  there  are  many  systems  in  use  by  the  Army  to  conduct  training  and  training  support  functions,  this 
search  and  summary  presents  highlights  of  a  few  of  the  more  widely  used  systems.  We  researched  and 
summarized  the  AAR  and  THP  capabilities  for  representative  systems  (*  systems  presented  are 
asterisked).  We  presented  these  systems  to  provide  the  backgroxmd  needed  to  determine  the  composition 
of  effective  THPs.  Finally,  we  summarized  the  few  papers  in  the  literature  that  address  any  type  of 
evaluation  of  the  effectiveness  of  THPs.  The  results  of  the  search  are  summarized  in  Volume  I  of  this 
report,  and  the  literature  search  results  including  citations  and  abstracts  are  contained  in  Volume  H. 
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3.  FINDINGS 


The  findings  are  presented  in  terms  of  the  purpose,  contents,  and  other  issues  surrounding  THP  content; 
sample  AAR  and  THP  systems  that  illustrate  today’s  THP  development  capability;  and  research  findings 
on  the  effectiveness  of  THPs. 

3-1  Current  THPs 

After  highlighting  the  purpose  and  typical  content  of  today’s  THPs,  we  will  discuss  differences  between 
THPs  based  on  the  organizational  level  of  participants  and  whether  participation  was  live  versus 
simulated. 

3.1.1  Purpose 

As  discussed  in  the  THP  definition  (see  Section  1.2)  the  Army  uses  THPs  in  the  following  roles: 

■  Plan  future  training  (JRTC,  1998a) 

■  Support  Lessons  Learned  from  AARs 

■  Provide  feedback  on  important  issues  not  discussed  during  AARs  (i.e.,  because  of  time 
limitations  at  the  training  exercise  site) 

■  Identify  and  analyze  data  trends  across  exercises  (Meliza  1995) 

The  various  training  environments  (live,  simulation,  and  constructive)  all  attempt  to  use  the  THP  in 
these  roles.  Reinforcing  lessons  learned  during  the  exercise,  the  AAR  is  designed  to  emphasize  the 
important  factors  affecting  the  exercises,  such  as  what  happened,  why  it  happened,  and  how  can  it  be 
corrected  (DA,  1997b).  After  the  AAR  has  been  presented,  the  THP  is  designed  to  allow  units  to  bring 
the  valuable  information  learned  at  the  exercise  back  to  home  base.  THPs  provide  units  with  feedback 
from  experienced  observers,  unit  performance  data,  and  videotapes  of  AARs,  allowing  units  to  train  on 
the  lessons  learned  from  the  exercises. 

Through  the  THP,  0/Cs  provide  a  wealth  of  information  on  each  unit’s 
performance  status.  Unfortunately,  however,  the  full  benefits  of  the  feedback 
information  are  not  realized.  At  the  unit  level,  brigade  and  battalion 
commanders  typically  move  to  other  assignments  before  they  design  training  to 
sustain  strengAs  and  correct  weaknesses;  and  their  replacements  are  frequently 
disinclined  to  work  through  previous  THPs  to  extract  the  detailed  evaluations 
they  contain.  (Ford,  Huffman,  &  Green,  1996,  pi) 

3.1.2  Contents  of  current  THPs 

At  this  point  in  time  the  AAR  process  has  not  been  standardized.  As  stated  in  the  final  coordination 
draft  of  the  STAARS  master-plan. 

Today,  there  are  no  standard  Army  system  protocols  to  capture  AAR 
data/information.  To  support  AARs,  many  training  systems  can  capture,  store, 
and  retrieve  relevant  data/  information.  In  some  cases,  stand-alone  AAR  systems 
were  developed  in  conjunction  with  the  Training  Aids,  Devices,  Simulators,  and 
Simulations  (TADSS),  while  in  others,  AAR  systems  were  added  later.  Most 
current  AAR  systems  facilitate  training  feedback  to  the  unit.  (DA,  1997b,  p.  1) 
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This  lack  of  standardization  also  affects  the  THPs  produced.  Even  so,  THPs  tend  to  consist  of  some 
common  components,  including 

■  objective  data, 

■  subjective  data,  and 

■  video  and  audio  portions. 

Examining  the  various  THE  systems,  UDRI-HFG  identified  sub-items,  which  often  occur  in  THPs,  falling 
roughly  into  the  broad  categories  listed  below: 

>  Objective  data 

•  Ordinance  measures 

■  Shots  fired 

■  Hits 

■  Misses 

■  Near  Misses 

■  Kills 

•  Casualty  reports 

-  Killed 

■  Wounded 

■  Ratio  Friend/Enemy  kills 

•  Location  data 

■  Map  coordinates 

■  Proper  formation 

■  Proper  intervals 

■  Movement  discipline 

■  Proper  turret  and  hull 
orientation  (armor  units) 

>  Subjective  data 

•  Notes  on  0/C  interference  during 

force-on-force  engagement 

■  Exercises  stopped  and  reran  due 
to  mid-exercise  coaching 

■  O/C  assessed  casualties 

■  Notes  on  proper  radio 
procedures 

■  Any  areas  not  covered  by  other 
sources 

3. 1.3  Organizational  Level  of  THP 

The  information  in  each  THP  differs  according  to  the  unit’s  organizational  level  (e.g.,  A  command 
post  THP  will  not  be  concerned  with  the  deployment  of  individual  tanks  or  even  platoons,  but  will  need  to 
be  informed  on  the  effectiveness  of  the  deployment  of  higher-level  units.  Conversely,  platoons  are  not 
concerned  with  the  operations  within  the  command  post;  therefore  the  types  of  information  need  to  be 
tailored  to  the  unit.)  The  Army’s  goal  is  to  adjust  the  information  in  a  THP  to  the  level  that  the  unit 
requires  for  the  exercise  involved,  but  to  keep  tire  information  in  THPs  consistent  across  the  different 
types  of  training.  As  stated  in  the  STAARS  Action  Plan,  AARs  and  THPs  should  contain  “standardized 
products  generated  in  or  a  combination  of  the  live,  virtual,  and  constructive  environments  will  be  identical 
or  nearly  identical  for  each  echelon  receiving  the  AAR.”  (DA,  1997b)  (See  Figure  1.) 


•  O/C  simunary  comments 

■  Overall  strengths 

■  Overall  wealmesses 

■  Unit  Attitude 

■  Subjective  comments 

■  Any  points  not  covered  by  other 
sources 

>  Video  and  Audio  Portions 

•  Communications  logs 

■  Contact  reports 

■  Spot  reports 

■  Proper  reporting  procedures 

■  Unit  crosstalk 

•  Videotapes,  video  capture,  and 

video  logs 

■  Videotapes  of  AAR  conducted 
throughout  exercise 

■  Videotapes  of  2-D  or  3-D  maps 
from  simulation 

■  Videotapes  of  engagements 
(some  force-on-force  areas) 

■  Video  capture  or  replay  from 
computer  simulation 
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FIGURE  1.  Information  Consistency  in  THPs 

(Source;  Annex  F,  Standard  Army  After  Action  Review  System,  to  OPORD  1-95  Department  of  Army,  1995 
Avaiiable:  http://www-dcst.monroe.army.mii/wfxxi/opanx-f2.htm) 

3.1.4  Live  vs.  Simulation:  Differences  in  THP  contents 

Several  differences  between  live  versus  simulated  training  environments  have  been  docxunented  in 
the  literature  that  may  have  a  bearing  on  AAR  and  THP  content  and  0/C  actions.  This  section  highlights 
these  considerations.  As  indicated  in  this  quote  from  Annex  F,  OPORD  1-95  (below),  the  ultunate  goal  of 
the  Army  is  to  create  AAR  products  that  are  consistent  across  the  training  spectrum  (live,  virtual, 
constructive  and  STOW),  it  appears  that  differences  still  exist. 

Clearly,  the  data  necessary  to  support  an  infantry  company  (Co)  AAR 
will  be  vastly  different  than  that  necessary  for  a  division  level  AAR; 
however,  there  is  no  reason  that  all  company  level  AAR  products  caimot 
be  standard,  no  matter  whether  the  training  event  takes  place  at  the 
National  Training  Center  (NTC)  or  in  a  simulation.  (DA  1995b,  Pg.  1) 


3.1. 4.1  Data 


The  data  collected  in  the  three  training  enviromnents  could  differ  in  type  and  volume.  The  data 
available  in  the  virtual  environment,  being  automated,  is  much  more  comprehensive  with  respect  to  each 
entity  in  the  exercise  for  example,  gun  tube  orientation  is  readily  available  in  the  virtual  world,  but  may 
not  as  readily  available  in  the  real  world  (Meliza,  1996  pg.  19). 

Information  about  the  status  of  entities,  including  information  about 
where  the  gun  tube  of  a  tank  is  pointed  at  a  specific  moment,  is  readily 
available  and  accessible  in  the  virtual  environment.  In  the  live 
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environment,  update  rates  on  such  aspects  of  entity  status  as  vehicle 
location  are  slower  than  in  the  virtual  world,  and  some  types  of 
information  (speed  of  entities  and  orientation  of  gun  tubes)  are  imlikely 
to  be  provided  in  the  near  future.  (Pg.  19). 


3.1. 4.2  Role  of  OIC 

Another  difference  between  the  live  exercises  and  simulation  environments  is  the  role  played  by 
the  0/C.  In  the  virtual  environment  the  O/C  records  notes  on  how  luiits  progress  through  established 
training  tables,  but  has  little  interaction  with  the  conduct  of  the  training  table,  other  than  changing  the 
difficulty  level  as  the  situation  dictates.  In  the  live  environment  the  O/C  plays  a  much  more  active  role, 
evaluating,  controlling,  and  interacting  with  participants  throughout  the  exercise.  By  contrast,  in  live 
simulations,  the  O/C  may  need  to  intervene  for  actions  that  the  simulation  will  perform  automatically. 
For  example,  the  O/C  may  need  to  “kill”  participants  in  a  live  exercise  when  they  are  “protected”  by 
artificial  barriers,  thereby  directly  affecting  the  exercise  outcome,  as  seen  in  this  example  of  MILES 
lasers  in  action  in  the  live  enviromnent  requiring  active  intervention  by  0/Cs. 

MILES  lasers  will  not  penetrate  minor  obstructions.  Tree  leaves  ("tree- 
leaf  defilade")  will  obstruct  the  laser.  Firing  positions  with  berms 
("MILES  berms")  that  are  inadequate  to  stop  penetration  by  real 
ordnance  will  stop  a  MILES  laser  beam.  Smoke  and  dust  precludes  the 
effectiveness  of  the  laser  and  may  preclude  engagements  at  maximum 
range.  For  safety,  JRTC  rules  of  engagement  preclude  the  use  of  MILES 
by  dismoimted  soldiers  for  close-in  engagements  at  less  than  10  meters. 

OCs  manually  perform  exercise  control  using  laser  pistols  (control  guns) 
in  those  instances  where  MILES  fidelity  limitations  or  safety  preclude 
automatic  casualty  and  battle  damage  assessments.  (Brown,  Nordyke, 

Gerlock,  Begley,  &  Meliza,  1997,  pg.  16) 

As  technology  improves,  the  differences  in  data  collection  between  the  enviroiunents  may  become 
smaller  and  the  Army’s  goal  of  AARs  and  THPs  that  are  consistent  across  enviromnents  might  become 
reality. 

3.2  Summaries  of  Selected  Systems 

To  give  the  reader  a  flavor  of  the  functions  provided  by  today’s  AAR  and  THP  systems  that  can  be  used 
to  develop  improved  THPs,  we  have  highlighted  capabilities  of  the  following  systems:  Unit  Performance 
Assessment  System  (UPAS),  Automated  Training  Analysis  and  Feedback  System  (ATAFS),  Training 
Analysis  Repository  and  Graphical  Evaluation  Toolset  (TARGET),  and  National  Training  Center 
Instrumentation  System  (NTC-IS).  These  systems  illustrate  the  capabilities  that  are  now  available  for 
THP  development. 
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3.2. 1  Unit  Performance  Assessment  System  (UPAS) 

A  pioneer  AAR  support  system,  UPAS  is  the  &st-generation,  automated,  personnel-computer  (PC)- 
based  AAR  preparation  aid  that  can  also  support  THPs.  It  facilitates  the  creation  of  AAR  and  THP  aids 
by  automatically  collecting  selected  data  generated  during  an  exercise.  The  AAR  aids  created  for  UPAS 
have  been  incorporated  into  the  next  generation  of  AAR  and  THP  preparation  systems. 

UPAS  can  migrate  scenes  from  an  exercise  into  an  AAR  or  THP,  enabling  the  unit  to  continue 
training  on  the  lessons  learned  diuing  the  exercise.  UPAS  operates  “on  top  of’  the  Simulation 
Networking  (SEMNET)  system  in  place.  UPAS  collects  data  from  the  SIMNET  exercise,  then  after  the 
data  is  collected,  can  be  used  in  stand-alone  mode  (without  needing  further  SIMNET  interface)  to  assist  in 
evaluating  unit  performance.  UPAS  is  designed  to 

•  collect  data  broadcast  over  the  network  (vehicle  location,  vehicle  status,  and  firing  events); 

•  filter  and  organize  the  data  to  support  rapid  analysis; 

•  load  data  into  a  relational  database  patterned  after  the  National  Training  Center  (NTC)  database; 

•  integrate  broadcast  data  with  imit  planning  and  terrain  data;  and 

•  provide  graphic  and  tabular  displays  of  data  to  support  unit  performance  analysis  and 
performance  feedback.  (Meliza  &  Tan,  1995,  pg.  1) 

UPAS  facilitates  the  creation  of  AAR  and  THP  by  displajdng  data  in  a  manner  that  supports 
animated  figures,  static  figures,  and  tables  that  can  be  used  to  illustrate  key  training  points  to  exercise 
participants.  These  data  are  shown  in  several  different  displays:  Plan  View  Display,  Battle  Flow  Chart, 
Battle  Snapshots,  Fire  Fight  Display,  Exercise  Timeline,  and  graphs  and  tables. 

All  available  data  in  the  simulation  environment  could  be  included  in  an  AAR  or  THP;  however,  the 
challenge  is  to  limit  data  presented  to  that  most  essential  for  training.  Most  data  needed  can  automatically 
be  loaded  from  the  simulation  into  UPAS;  however,  there  are  some  exceptions.  During  an  exercise  the 
O/C  may  need  to  annotate  individual  start/stop/restart  times  to  be  included  in  the  AAR  and  THP.  Also, 
the  inter-unit  communications  used  during  the  simulation  are  not  available  to  the  O/C  in  computerized 
form.  Therefore,  “A  data  collector  will  be  required  to  collect  communications  data  and  load  it  into  UPAS 
in  order  to  use  the  communications  portion  of  the  Exercise  Timeline.”  (Meliza  &  Tan,  1995,  pg.  65) 
Consequently,  the  O/C  must  keep  a  hand-written  log  of  all  pertinent  communications.  The  types  of 
communications  are  added  onto  the  Event  Timeline.  In  addition  to  making  O/C  notes  on  the  unit  s 
communications,  the  O/C  may  also  input  notes  to  be  added  to  each  of  the  different  types  of  displays  for 
inclusion  into  the  AAR  and  THP.  The  UPAS  capability  for  creating  THPs  consists  of  a  utility  to  capture 
screen  images  of  pertinent  maps  or  AAR  slides  that  can  be  loaded  onto  a  disk  for  reviewing  at  the  unit  s 
home  station. 


3.2.1. 1  Plan  View  Display 

The  Plan  View  Display  (see  Figure  2)  shows  the  exercise  from  a  two-dimensional  overhead 
perspective.  The  exercise  area  is  displayed  as  a  grid  map  with  terrain  features  such  as  elevation,  trees, 
water,  roads,  and  buildings.  Participating  units  are  shown  on  the  grid  map  as  picture  icons  indicating  umt 
type  (e.g.  ground  vehicle,  dismoimted  infantry,  rotary  wing,  and  conventional  aircraft),  with  the  Unit 
Control  Measures  listed  in  yellow  text.  Unit  icons  are  also  color-coded  corresponding  to  status  (e.g. 
operational  or  destroyed).  The  Plan  View  Display  may  be  an  animated  display  showing  the  time-flow  of 
the  engagement  or  a  snapshot  of  any  moment  in  the  exercise. 
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FIGURE  2.  Plan  View  Display  (UPAS  Sample) 

(Source:  SIMNET  Unit  Performance  Assessment  System  (UPAS) 
Version  2.5  User's  Guide,  Meiiza  &  Tan,  1995) 
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3.2.1.2  Battle  Flow  Chart 


The  Battle  Flow  Chart  (see  Figure  3)  is  a  time-dependent  trace  of  a  unit’s  movement  during  the 
exercise.  This  display  allows  the  unit  to  see  how  and  where  individuals  moved  during  the  exercise.  This 
allows  the  units  to  evaluate  whether  proper  maneuver  techniques  were  employed. 


Start  Tine:  150000  Battle  Flows  Final  Tine!  153450 

A  <PLT  2>  ;  B22  B  <PLT  2>  :  B23 

C  <PLT  2/’LEAB>  :  B21  D  <PLI  2/SCRG  >  :  B24 


FIGURE  3.  Battle  Flow  Chart  (UPAS  Sample). 

(Source;  SIMNET  Unit  Performance  Assessment  System  (UPAS) 
Version  2.5  User's  Guide,  Meiiza  &  Tan,  1995) 
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3.2. 1.3  Battle  Snapshots 

Battle  Snapshots  (see  Figure  4)  show  vehicle  positions  and  gun  tube  orientation  (if  applicable)  as 
specified  time  points  in  the  simulation.  This  feature  provides  units  with  position  information  as  well  as 
orientation  information.  The  Snapshot  also  has  the  capability  to  indicate  line-of-sight  between  friendly 
and  enemy  vehicles.  This  capability  can  be  used  to  assess  the  cover  and  concealment  offered  by  a  imit's 
halt  positions,  battle  positions,  and  routes.  (Meliza  &  Tan,  1995,  Pg.  4) 


FIGURE  4.  Battle  Snapshot  (UPAS  Sample). 

(Source:  SIMNET  Unit  Performance  Assessment  System  (UPAS) 
Version  2.5  User's  Guide,  Meiiza  &  Tan.  1995) 


3.2. 1.4  Fire  Fight  Display 

Fire  Fight  Displays  (see  Figure  5)  allow  the  units  and  trainers  to  visualize  how  the  exercising  unit 
covered  the  battlefield  with  direct  and  indirect  fires.  This  display  is  animated  to  cover  a  period  of  time 
selected  by  the  0/C.  Shot  lines  are  shown  extending  from  the  originating  vehicle  to  the  point  of  impact. 
Hits  and/or  kills  are  shown  as  green  lines  and  misses  as  white.  Indirect  fires  are  shown  with  a  rectangle. 
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FIGURE  5.  Fire  Fight  Dispiay  (UPAS  Sampie). 

(Source:  SIMNET  Unit  Performance  Assessment  System  (UPAS) 
Version  2.5  User’s  Guide,  Meliza  &  Tan,  1995,  p.  61) 


3.2.1 .5  Exercise  Timelines 

Exercise  timelines  provide  a  time-correlated  picture  of  what  happened  during  the  exercise.  The 
exercise  timeline  shows  how  all  exercise  events  relate  to  each  other  in  a  time  sense. 

3.2.2  Automated  Training  Analysis  and  Feedback  System  (ATAFS): 

L6&M  Associates,  now  Advancia  Corporation,  developed  ATAFS  via  a  series  of  Small  Business 
Research  Innovation  (SBIR)  contracts  monitored  by  the  Army  Research  Institute  (ARI)  and  flmded 
through  STRICOM  and  ARPA.  ATAFS  was  created  to  address  shortcomings  in  Ae  UPAS  systems  such 
as  the  time  required  to  prepare  displays,  the  inability  to  link  radio  transmission  data  to  UPAS  data,  and  no 
support  for  automatic  data  display  selection  (Brown,  Wilkinson,  Nordyke,  Hawkins,  Robideaux,  & 
Huyssoon,  1996,  and  ARI,  1998).  To  facilitate  AAR  preparation,  ATAFS  employs  a  knowledge  base  to 
guide  the  creation  of  AAR  aids.  ATAFS  uses  a  combination  of  automated  data  monitoring,  for  events 
with  discrete  start  and  end  times  and  0/C  observations,  for  events  without  easily  identifiable  start  and  end 
times,  to  create  its  AAR  aids.  The  AAR  aids  created  by  ATAFS  include: 

■  plan-view  animation, 

■  snapshots, 

■  battle  flow, 

■  firefight  display,  and 

■  statistical  graphs,  and  tables. 

Additionally,  ATAFS  can  synchronize  radio  commimications  with  the  animated  plan-view.  To  further 
aid  O/Cs  in  AAR  production,  the  O/Cs  are  able  to  add  AAR  discussion  points  directly  on  the  ATFAS- 
generated  AAR  aids  (Figine  6).  ATAFS  stores  the  AAR  aids  and  allows  the  user  to  edit  these  aids  before 
they  are  used  to  support  AARs  or  loaded  onto  a  videotape  for  use  in  the  THP  (Nordyke,  John,  personal 
communication,  10  Nov,  1998). 
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Actions  on  Initial  Contact 

start  Time:  00:05:45  Battle  Drill  Ordered 

End  Time:  00:06:40  Spot  Report  to  Co/Tm  Cdr  00:06:40 
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FIGURE  6.  AAR  Aid  with  Discussion  Points  (ATAFS  Sample). 

(Source:  Brown,  Wilkinson,  Nordyke,  Hawkins,  Robideaux,  SHuyssoon,  1996,  Pg.  11) 

Used  by  permission  ofAdvancia  Corporation.) 

3.2.3  Training  Anaiysis  Repository  and  Graphicai  Evaiuation  Tooiset  (TARGET) 

Initial  work  on  Virtual  Training  Packages  was  conducted  by  Burnside,  Leppert,  and  Myers,  (1996)  under 
the  auspices  of  the  US  Army  Research  Institute  for  the  Behavioral  Sciences  (ARI),  which  produced  the 
Virtual  Training  Program  Orientation  Guide.  The  Virtual  Training  Repository  (VTR)  was  developed 
with  Defense  Advanced  Research  Projects  Agency  (DARPA)  funding  under  the  Computer-Aided 
Training  and  Education  Initiative  (CAETI)  program.  TARGET  (see  Figure  7)  was  developed  as  a  VTR 
tool  to  effectively  analyze  and  visualize  data  from  individual  simulation  exercises,  and  across  exercises. 
TARGET  provides  tools,  techniques  and  processes  to  conduct  longitudinal  studies  across  several 
exercises.  TARGET  uses  graphic  tools  to  display  different  Measures  of  Performance  (MOPs).  For 
example,  TARGET  tools  include  the  following:  bar  charts,  event-based  timelines,  a  set  of  statistical 
analysis  tools,  a  2-D  Battle  Viewer,  and  a  3-D  enhanced  Battle  Viewer  for  exercise  playback  (Loughran, 
Johnson,  Kappel,  &  Stahl,  1997). 
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FIGURE  7.  THP  Screen  (TARGET  Sample). 

(Source:  Loughrarn  Johnson,  Kappel,  &  Stahl,  1997) 

TARGET  uses  THPs  from  the  units  exercising  in  the  Virtual  Training  Program  (VTP),  developed  by 
Burnside,  Leppert,  and  Myers,  (1996).  The  VTP  THPs  consist  of  a  Microsoft  Excel  spreadsheet  indexed 
by  O/C  team  (see  Figure  8).  Each  exercising  platoon  is  provided  with  a  set  of  prescribed  training  tables, 
with  each  table  focusing  on  opportunities  to  train  specific  skills.  For  example,  a  particular  table  may  call 
for  the  platoon  to  conduct  a  “hasty  defense”  and  perform  all  the  subtasks  associated  with  a  “hasty 
defense.”  During  the  training  exercise  the  tables  progress  in  complexity  (“crawl-walk-run”  philosophy). 
Additionally,  the  O/C  can  tailor  the  progression  from  table  to  table  as  the  skill  levels  increase.  This 
allows  each  platoon  to  train  problem  tasks  that  are  imique  to  the  unit.  Regardless  of  the  number  of  times 
the  unit  performs  each  table,  each  unit  is  provided  only  one  THP.  THPs  have  unit  performance  rated  as 
one  of  the  following: 

I  =  Improve, 

S  =  Sustain,  or 
N=  Not  Trained. 

O/C  comments  are  provided  along  with  the  task  rating.  In  addition  to  the  comments  listed  in  the  THP 
table,  the  0/Cs  provide  a  comprehensive  summary  of  general  O/C  comments  and  areas  to  improve  and 
sustain.  (See  Figure  9  for  a  sample  of  freeform  O/C  comments.) 
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Task  and  Subtask 

Page* 

Combat 

Function 

Code 

PAB 

2 

PAB 

3 

PAC 

1 

PAC 

2 

Final 

Comments 

17-3-0221  -  Execute 
Actions  on  Contact 

- 

- 

■ 

■ 

■ 

- 

■ 

- 

Tank  commander  who 
first  observes  the 
enemy  takes  action.  * 

5-85 

A5 

I 

I 

I 

I 

I 

"Individual  crews  acted 
on  contact  properly,  but 
no  reports  were  going 
higher  to  the  platoon 
leader  or  commander.  By 
the  time  reports  got  to  the 
commander,  the  platoon 
was  decisively  engaged  or 
destroyed." 

"The  remainder  of  the 
platoon,  upon  seeing 
other  vehicle(s) 
engage  and  hearing 
the  contact  report, 
takes  action. " 

5-85 

A6 

s 

s 

s 

s 

s 

- 

Platoon  sends  a 
contact  report  to  the 
commander. 

5-86 

C5 

s 

s 

s 

i 

s 

1 

s 

- 

Platoon  sends  a 
complete  spot  report 
to  the  commander.  * 

5-86 

C6 

I 

I 

I 

I 

I 

"Throughout  the  exercise, 
the  platoon  leader  had 
problems  with  acquiring 
the  information  from  his 
TCs  to  send  an  accurate 
situation  report  to  the 
commander." 

Platoon  leader  directs 
a  platoon  battle  drill. 

5-86 

C9 

s 

s 

s 

s 

s 

- 

FIGURE  8.  THP  Spreadsheet  (TARGET  Sample) 

(Available:  httD://www.ida.ora/DIVISIONS/csed/vtr/aboutData.htmn 
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Observer/Controller  Comments  -  08p51021 

1.  O/C.  SFCMAHNS 

2.  General.  1st  Platoon  _  conducted  the  Virtual  Training  Program  (VTP)  familiarization  course,  a 
modified  fundamental  table  and  platoon  level  offensive  missions.  The  platoon  was  scheduled  to  conduct 
the  familiarization  table  and  several  offensive  exercises.  During  the  introductory  briefing  and  subsequent 
inquiries  with  platoon  members  it  was  determined  that  the  platoon  was  not  prepared  to  conduct  the 
missions  planned  without  some  rudimentary  training.  Few  members  of  the  platoon  were  familiar  with  the 
platoon’s  Standard  Operating  Procedure  (SOP).  Only  two  members  had  been  in  simulators.  The  platoon 
had  not  practiced  movement  techniques  recently.  After  confemng  with  the  platoon  leader  and  the  tank 
commanders  it  was  decided  that  we  would  conduct  the  familiarization  course,  followed  by  some  basic 
tactical  movement  subtasks,  before  starting  the  offensive  exercises 

3.  Strengths  to  Sustain. 

Cross-talk.  After  some  coaching  the  platoon  became  very  efficient  at  using  cross-talk  to  enhance  their 
performance. 

Attitude.  Even  though  the  platoon  had  to  repeat  tasks  several  times,  they  maintained  a  positive 
attitude  and  open  minds. 

Crew  duties.  Tank  crewmembers  worked  as  a  team. 

4.  Areas  Improved  during  SIMNET  Training. 

Movement  Techniques.  The  platoon  learned  some  very  valuable  lessons  about  maneuvering  as  a  platoon 
and  have  a  good  base  on  which  they  can  build. 

Executing  battle  drills.  Battle  drills  became  smoother  by  the  end  of  the  session,  but  still  need  work. 

5.  Areas  to  Improve. 

Command-Control-Communication.  The  platoon's  tank  commanders  have  varying  degrees  of 
experience  in  armor  platoon  tactics.  The  platoon  leader  has  to  ensure  that  his  tank  commanders  are  very 
familiar  with  the  tactical  SOP.  The  platoon  never  fought  as  a  platoon,  but  as  four  separate  tanks.  The  PSG 
was  slow  to  send  spot  reports  to  the  commander.  Reports  sent  were  not  in  the  proper  format. 

Battle  DrillsIActions  on  Contact.  The  platoon  leader  had  a  good  concept  of  what  actions  needed  to 
take  place.  He  had  a  difficult  time  issuing  the  proper  commands  in  a  timely  fashion.  There  was  confusion 
initially  concerning  the  proper  format  and  actions  to  be  taken  when  the  platoon  made  contact  with  aircraft. 

Knowledge  of  Platoon  SOP.  This  platoon  has  the  intelligence  base  and  desire  to  become  an  effective 
arm  of  combat  decisiveness  with  proper  training.  The  platoon  must  spend  more  time  studying  FM  17-15 
Tank  Platoon  and  FKSM  17-15-3,  Tank  Platoon  SOP.  All  platoon  members  must  know  and  understand 
their  platoon  SOP. 


FIGURE  9.  O/C  Summary  Comments  (TARGET  Sample) 

(Available:  httD://www.ida.ora/DIVISIONS/csed/vtr/data/thp/html/02A60224.htmn 
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3.2.3.1  2-D  Battle  Viewer 


The  2-D  Battle  Viewer  (Figure  10)  is  a  TARGET  application  that  provides  the  capability  to  play 
back  the  exercise  from  an  overhead  point-of-view.  A  2-D  terrain  map  with  orientation  grids  (currently 
the  National  Training  Center)  is  displayed.  The  details  of  the  terrain  features  (e.g.,  elevation,  soil  types, 
and  roads)  are  derived  from  Modular  Semi-Automated  Forces  (ModSAF)  files  and  are  graphically 
displayed.  Each  vehicle’s  route  is  plotted  as  a  solid  line  (Blue  platoon’s  vehicles  are  individually  color- 
keyed)  on  the  terrain  map.  In  addition  to  the  routes,  all  shots  are  plotted  as  dotted  color-keyed  lines.  The 
results  of  each  shot  are  also  indicated.  In  the  viewer  the  map  may  also  be  scaled,  and  any  of  the  marker 
features  may  be  toggled  on  or  off.  To  compare  various  trials  of  Ae  same  training  table,  the  2D  Battle 
Viewer  is  capable  of  replaying  several  trials  at  the  same  time. 


FIGURE  10.  Screen  Image  of  2D  Battle  Viewer  (TARGET). 

(Source:  Loughran,  Johnson,  etal,  1997) 

3. 2. 3. 2  3-D  Enhanced  Battle  Viewer 

Advancing  technology  has  provided  low-cost  personal  computers  with  the  capability  of 
displaying  graphics  that  had  previously  only  been  available  on  high-end  UNIX  computers.  Along  with 
this  capability,  TARGET  has  developed  a  prototype  3-D  enhanced  Battle-Viewer  that  utilizes  Virtual 
Reality  Modeling  Language  (VRML)  to  display  3-D  images  of  the  exercise.  Units  are  able  to  access  and 
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replay  the  simulation  throu^  the  3-D  Battle  Viewer  from  their  home-station  via  the  Internet.  Currently 
the  3-D  enhanced  Battle  Viewer  consists  of  a  VRML  plug-in  for  web  browsers.  This  allows  the  person 
viewing  the  replay  to  “move”  around  the  scene  and  watch  the  engagement  from  different  points  of  view. 
The  principals  in  the  simulation  replay  are  generic  polygon  vehicles  showing  both  hull  and  gun  tube 
orientation.  As  the  scenario  progresses,  any  firing  conducted  by  the  tanks  is  displayed  as  a  shot  leaving 
the  tube.  In  addition  to  the  fire  visualization,  firing  statistics  (total  rounds  fired,  hits  and  misses)  are 
displayed  above  the  image  of  the  tank. 

3.2.4  National  Training  Center  Instrumentation  System  (NTC-iS) 

In  the  late  70s  and  early  80s  the  Army  identified  a  need  to  train  in  a  live,  force-on-force  environment. 
The  National  Training  Center  (NTC)  at  Ft.  Irwin,  CA  was  developed,  in  1982,  as  a  direct  result  of  this 
realization.  Heavy  units  deploy  to  the  NTC  to  conduct  the  most  realistic  training  possible,  focusing  on 
medium-to-high-intensity  conflict  in  the  desert,  that  the  US  military  has  to  offer.  In  contrast  to  ModSAF, 
the  NTC  allows  units  to  conduct  training  against  live,  thinking  foes,  trained  to  fight  according  to  existing 
enemy  doctrine.  The  success  of  the  NTC  led  to  the  development  of  other  live  training  facilities  tailored  to 
different  units  and  missions,  e.g.,  the  Joint  Readiness  Training  Center  (JRTC)  at  Ft.  Polk,  used  by  light 
forces,  and  the  Combat  Maneuver  Training  Center,  Hoenfels,  Germany,  which  allows  xmits  to  train  in 
European  scenarios. 

The  NTC  THP  consists  of  two  portions,  a  written  portion  and  a  videotape  portion.  A  survey 
conducted  by  Fobes  and  Meliza  (1988)  discovered  that,  at  the  time  it  was  conducted,  the  written  portion 
of  die  THP  was  considered  ineffective  in  supporting  sustainment  training  at  the  unit’s  home  station. 
“Commanders  reported  the  written  portion  of  the  THP  does  not  effectively  support  post-rotational 
corrective  and  sustainment  training  at  home  station”  (Fobes  &  Meliza,  1988).  On  the  other  hand,  the 
videotape  portion  of  the  THP  was  considered  a  useful  tool  for  home  station  sustainment  as  well  as  pre- 
rotational  planning. 

In  line  with  these  observations  the  NTC-IS  is  designed  to  increase  the  effectiveness  of  the  THP  by 
adding  several  multimedia  aspects  to  the  package.  “The  multimedia  AARs  and  the  THP  of  unit 
performance  are  key  components  of  the  combat  training  center  experience.  The  NTC-IS  will  collect, 
process,  analyze,  and  display  training  performance  data  to  support  the  preparation  of  AARs  and  the  THP. 
The  miiltimpiflia  AARs  and  THP  are  valuable  sources  of  data  for  the  Amy  Lessons  Learned  Program.” 
(DA,  1995a).  The  NTC-IS  operational  requirement  document  listed  the  necessary  AAR  and  THP 
capabilities  as  follows: 

Infomation  presentation  capabilities  must  support  activities  to  present  training 

perfomance  infomation  to  rotational  units  in  the  multimedia  AARs  and  THPs. 

The  NTC-IS  must: 

■  present  prepared  training  perfomance  feedback  during  multimedia  AARs 
conducted  in  mobile  facilities  in  multiple  field  locations; 

■  present  prepared  training  performance  feedback  during  multimedia  AARs 
conducted  in  fixed  facilities  in  the  cantonment  area; 

■  replay  prepared  voice,  video,  and  digital  training  perfomance  information  in 
the  multimedia  AAR  facilities  on  command; 

■  replay  the  exercise  history  in  the  multimedia  AAR  facilities  on  command; 

■  record  audio  and  video  portions  of  multimedia  AARs  (both  system-and 
audience-generated)  for  inclusion  in  the  THP;  and 

■  prepare  the  THP  for  delivery  to  the  rotational  unit  in  hard  copy  and 
removable  electronic  media.  (DA,  1995a) 
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3.3  Research  on  THP  Effectiveness 

We  discovered  only  one  published  study  dining  the  period  1987  —  1998  that  documented  findings 
regarding  the  effectiveness  of  military  training  AAR  THPs.  THPs  seem  to  have  been  overlooked  by  the 
training  evaluation  researchers,  for  the  most  part.  The  single  study  covering  THP  effectiveness  was 
limited  to  THPs  developed  at  the  NTC.  Fobes  and  Meliza  (1988)  conducted  a  survey  of  commander  of 
most  echelons  from  three  divisions  rotating  tiirough  the  NTC.  These  commanders  were  queried  on  THP 
effectiveness  during  a  two-month  period  during  1987.  Fobes  and  Meliza  concluded  from  this  limited 
NTC  study  that  the  written  portion  of  the  THP  is  almost  never  used  due  to  its  length  and  lack  of  specific 
recommendations,  while  the  video  portion  was  used  for  several  purposes. 

Commanders  [surveyed]  reported  that  written  portion  of  the  THP  does 
not  effectively  support  post-rotational  corrective  and  sustaimnent 
training  at  home  station.  Users  indicated  this  material  is  too  extensive 
and  complex,  contains  many  inconsistencies,  and  lacks  specific 
recommendations  for  corrective  training.  (Fobes  &  MeUza,  1988,  p.  viii) 

Although  AAR  and  THP  technology  has  changed  greatly  since  this  NTC  study  was  conducted,  the 
lessons  learned  are  probably  still  ^plicable  to  the  THP.  Recent  evidence  reinforces  the  conclusions  made 
in  that  study.  In  a  personal  communication  Julia  Loughran  on  the  DARPA  VTR  project  team  states 
"...  while  we  did  this  project,  we  heard  from  a  number  of  people  that  THPs  were  never  looked  at  again 
after  a  training  event.”  (Loughran,  Julia,  personal  communication,  December  30,1998).  Additionally, 
Ford,  Huffinan,  &  Creen  (1996)  state: 

Unfortunately,  however,  the  full  benefits  of  the  feedback  information  are 
not  realized.  At  the  unit  level  brigade  and  battalion  commanders 
typically  move  to  other  assignments  before  they  design  training  to 
sustain  strengths  and  correct  weaknesses;  and  their  replacements  are 
frequently  disinclined  to  work  through  previous  THPs  to  extract  the 
detailed  evaluations  they  contain. 

Even  analysts  who  are  highly  motivated  to  extract  information  from 
THPs  must  spend  much  of  their  time  structuring  the  information  to  make 
it  accessible.. . .  Much  of  the  information  that  is  available  in  the  THP  is 
not  suitable  for  quantitative  analysis  that  considers  several  rotations. 

(Pg-1) 

While  the  Ford,  Huffinan,  &  Creen  (1996)  study  was  not  explicitly  studying  the  effectiveness  of  the  THP, 
its  observations  tend  to  confirm  the  shortcoming  identified  by  Fobes  and  Meliza  (1988).  Likewise,  in 
stating  reasons  for  commissioning  the  present  literature  review.  Dr.  Mona  Crissey  stated,  “In  all  my 
travels  [to  simulation  and  military  training  sites],  take-home  packages  of  past  exercises  . . .  seem  to  be  a 

(Crissey,  Dr.  Mona,  personal  communication,  August  6,  1998).  Dr.  Crissey 
had  hoped  that  this  literature  search  would  have  uncovered  other  studies  feat  would  point  fee  way  to 
improving  fee  THP.  Thus,  these  informal  comments  tend  to  validate  fee  need  to  conduct  a  study  of 
current  THPs  to  determine  how  they  can  be  improved. 

Shlecter,  Shadrick,  Bessemer,  &  Anthony  (1997)  reported  a  related  finding  through  their  examination  of 
fee  effects  of  units'  home-station  preparation  upon  their  Virtual  Training  Program  (VTP)  activities  and 
effects  of  fee  VTP  on  platoons  and  their  unit  leaders.  Specifically,  fee  participants  responses  indicated 
feat  fee  VTP  had  a  significant,  though  modest,  impact  upon  VTP-experienced  unit  leaders'  sense  of 
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confidence  in  their  own  and  their  unit's  tactical  proficiency.  The  instructors'  responses  suggested  that  the 
VTP  had  a  salient  effect  upon  the  tactical  skill  proficiency  of  the  sampled  unit  leaders.  Thus,  home-station 
preparation  did  have  an  impact  upon  the  training  participants'  VTP  performance. 
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4.  DISCUSSION  AND  CONCLUSIONS 


Concentrating  on  studies  conducted  since  1987,  this  search  and  summary  identified  only  one  document  in 
the  scientific  and  technical  (S&T)  literature  ejq)licitly  reporting  on  the  effectiveness  of  military  training 
AAR  THPs.  The  report  by  Fobes  and  Meliza  (1988)  summarizes  their  survey  of  commanders  from 
several  echelons  of  three  divisions  rotating  through  the  NTC  on  the  effectiveness  of  the  standard  NTC 
THP.  While  the  survey  is  on  THP  products  produced  with  the  technology  available  at  that  time,  and  the 
sample  size  and  location  are  limited,  the  observations  on  potential  improvements  to  THPs  may  still  be 
applicable.  Other  informal  observations  by  Ford,  Huffman,  &  Creen  (1996),  Loughran  (personal 
communications,  1998),  and  Crissey  (personal  communications,  1998)  tend  to  confirm  the  need  for 
improvement  of  THPs. 

Fobes  and  Meliza  (1988)  indicated  that  at  that  time  the  written  portion  of  THPs  did  not  satisfactorily 
address  performance  strengths  and  weakness.  Consequently,  THPs  did  not  provide  information  necessary 
to  identify  areas  for  corrective  training.  On  the  other  hand,  Fobes  and  Meliza  point  out  that  the  take-home 
videotaped  segments  were  widely  used.  At  that  time,  the  video  sections  were  used  from  brigade  through 
company  levels  and  were  frequently  borrowed  as  preparatory  training  for  units  planning  to  go  to  the  NTC. 
In  addition  to  being  used  by  the  reviewed  unit,  the  videotaped  segments  were  used  as  pre-rotation  training 
aids  for  other  units  scheduled  for  the  NTC.  The  interviewed  commanders  suggested  reducing  the  size  and 
complexity  of  the  THP.  They  suggested  condensing  the  written  portion  to  a  few  bullets  covering  mission 
outcome,  major  strengths  and  weaknesses,  critical  underlying  events,  and  specific  training 
recommendations.  The  only  change  to  the  visual  portion  suggested  was  the  inclusion  of  paper  copies  of 
the  AAR  briefing  slides. 

Going  further,  Fobes  and  Meliza  (1988)  suggested  a  more  structured  approach  to  the  written  portion  of 
the  THP,  with  each  mission  to  be  evaluated  according  to  each  echelon’s  Mission  Essential  Task  List 
(METL).  Since  each  METL  is  divided  into  critical  tasks,  mission  success  or  failure  can  be  traced  to  its 
critical  task.  This  type  of  mission  decomposition  allows  the  underlying  factors  for  each  mission  success 
or  failure  to  be  identified,  which  in  turn  allows  specific  training  recommendations  to  be  developed. 
Similarly,  Dr.  Crissey  (personal  communication  August  6,  1998)  observed  from  her  experience  visiting 
military  simulation  and  training  sites,  “What  commanders  really  use  is  the  results  of  the  task  evaluations 
as  the  start  point  for  planning  new  work  [training  initiatives].”  These  more  recent  observations  now  have 
the  benefit  of  operating  in  a  METL-oriented  training  environment,  as  Fobes  and  Meliza  had  suggested  in 
1988.  With  the  pervasive  use  of  METLs  for  training,  perhaps  the  time  for  a  reassessment  of  THP 
usefulness  has  now  arrived. 

Another  observation  of  this  search  and  summary  is  that  the  line  separating  AAR  and  THP  capabilities  is 
beginning  to  evaporate  with  the  advent  of  more-advanced  and  less-expensive  graphics  technology.  The 
multimedia  cq)abilities  of  the  THP,  which  were,  at  one  time,  only  available  on  high-end  graphics 
workstations  are  now  available  to  most  personal  computers,  and  therefore  available  to  more  of  the 
training  population.  This  allows  THP  systems  to  include  more  data  displays  for  easier  use  in  the  THP. 

The  WWW  will  also  play  a  role  in  advancing  the  capabilities  and  effectiveness  of  the  THP.  Internet- 
based  THP  systems,  such  as  the  TARGET  database  and  other  newer  systems  will  allow  any  authorized 
units  access  to  all  the  training  data  across  exercises,  not  just  their  own.  This  capability  will  allow  them  to 
see  how  their  unit  compares  with  others  participating  in  similar  exercises  and  to  examine  alternative 
solutions. 

Lacking  a  current,  comprehensive  study  of  THP  effectiveness,  and  since  considerable  effort  is  expended 
to  produce  THP  products,  it  seems  appropriate  to  conduct  a  new  study  of  THP  effectiveness  to  ensure 
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mayitniim  utility  from  this  important  potential  training  product.  This  proposed  study  on  the  effectiveness 
of  die  THP  could  be  conducted  via  a  structured  and  open-ended  survey  of  commanders  and  other  selected 
participants  from  luiits  participating  in  a  variety  of  exercises  (live,  constructive,  and  simulation).  All 
facets  of  the  THP  could  be  addressed  to  determine  the 

•  extent  of  use  for  the  various  types  of  THP  products, 

•  most  effective  methods  and  uses  of  THPs,  and 

•  desired  improvements  to  and  configurations  of  THPs  that  would  encourage  field  use. 
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